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Abstract 

The  robustness  of  a  subspace  generalized  likelihood  ra¬ 
tio  test  (GLRT)  detector  to  signal  mismatch  is  addressed 
for  data  conforming  to  the  generalized  multivariate  anal¬ 
ysis  of  variance  model.  This  model  assumes  a  deter¬ 
ministic  signal  of  known  form  in  the  presence  of  un¬ 
known,  colored,  Gaussian  noise.  The  subspace  GLRT 
compresses  data  into  a  lower-dimensional  subspace 
prior  to  detection.  It  is  shown  in  this  paper  that  a  sub¬ 
space  GLRT  reduces  the  performance  loss  due  to  mis¬ 
match  relative  to  that  of  a  non-subspace  detector. 

I.  Introduction 

A  common  signal  processing  problem  is  the  detection  of 
a  signal  having  a  known  form  in  the  presence  of  noise. 
This  problem  occurs  in  active  radar,  active  sonar,  com¬ 
munication  systems,  biomedical  stimulus  response,  and 
many  other  multi-sensor  or  time-series  applications. 
For  a  radar  array,  a  signal  form  is  typically  derived 
from  known  sensor  locations,  transmitter  location,  and 
frequency.  In  time  series  applications,  a  signal  form  is 
derived  from  the  relative  amplitudes  as  a  function  of 
time.  We  will  consider  as  unknowns  the  noise  covari¬ 
ance  matrix  and  the  signal  magnitude.  The  signal  is  ab¬ 
sent  when  this  magnitude  is  zero.  Note  that  this  detec¬ 
tion  problem  differs  from  a  direction  of  arrival  estima¬ 
tion  problem  where  the  task  is  to  ascertain  the  existence 
of  a  signal  of  unknown  form. 

Subspace  generalized  likelihood  ratio  test  (GLRT) 
detectors  have  been  shown  to  yield  strong  performance 
gains  over  full  data  space  detectors  (e.g.  [1]  and  [2]) 


when  the  number  of  data  vectors  is  limited.  These  sub¬ 
space  detectors  are  constructed  by  passing  data  through 
a  linear  ‘subspace’  (matrix)  transformation  prior  to  per¬ 
forming  signal  detection.  See  figure  1.  Performance  in 
this  case  is  measured  by  the  detector’s  probability  of 
detection  (PD)  for  a  fixed  probability  of  false  alarm 
(PFA).  A  subspace  detector  gains  this  performance  in 
three  primary  ways.  First,  a  priori  information  about 
the  interference  scenario  often  can  be  folded  indirectly 
into  the  detector  through  a  judicious  choice  of  subspace 
transformation.  Direct  inclusion  of  additional  informa¬ 
tion  into  the  full  data  space  model  usually  leads  to  an 
intractable  problem.  Second,  a  subspace  detector  has 
significantly  fewer  model  parameters.  This  means  that 
there  is  effectively  more  data  to  estimate  each  parameter. 
Hence,  the  estimates  tend  to  be  more  stable  which  im¬ 
proves  subspace  detector  performance.  Third,  because 
a  subspace  detector  has  fewer  parameters  to  estimate  it 
requires  less  data  to  exist.  In  many  cases,  a  subspace 
detector  can  exist  with  good  performance  when  insuffi¬ 
cient  data  exists  to  construct  the  full  data  space  detector. 
The  disadvantage  of  subspace  processing  is  that  most 
non-adaptive  transformations  will  tend  to  diminish  the 
signal-to-noise  ratio  (SNR).  A  loss  in  SNR  corre¬ 
sponds  to  a  loss  in  detection  performance. 

The  relative  advantage  of  subspace  detection  over 
full  data  space  detection  (when  T  =  I;^)  depends  almost 
entirely  upon  the  amount  of  available  data.  It  is  shown 
in  [1]  that  with  short  data  records,  statistical  stability 
dominates  detector  performance  and  subspace  detection 
is  preferable  to  full  data  space  detection  in  spite  of 
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Figure  1:  Block  Diagram  of  a  Non-Adaptive  Subspace  Detector. 
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typical  SNR  losses.  With  large  data  records,  SNR 
dominates  detector  performance  and  the  full  data  space 
detector  has  better  performance. 

This  paper  discusses  a  fourth  advantage  of  subspace 
detectors.  It  is  shown  that  subspace  GLRT  detectors 
are  more  robust  than  full  data  space  GLRT  detectors  to 
signal  model  mismatch.  Kelly  and  Forsythe  [4]  define 
signal  mismatch  as  the  effect  of  imperfect  knowledge  of 
the  form  of  signals  in  the  GLRT  model.  Robustness  is 
demonstrated  in  this  paper  by  showing  that  subspace 
processing  reduces  the  effect  of  mismatch  between  the 
assumed  signal  model  forms  and  true  signal  model 
forms.  These  reductions  occur  for  two  reasons. 
Primarily,  mismatch  is  reduced  when  data  is  trans¬ 
formed  into  a  subspace  because  linear  compression 
tends  to  diminish  the  differences  between  data  forms. 
Secondarily,  subspace  detectors  can  be  less  sensitive  to 
changes  in  SNR  that  result  from  erroneous  signal  model 
assumptions. 

For  notation,  bold  lower  and  upper  case  roman  let¬ 
ters  designate  column  vectors  and  matrices,  respec¬ 
tively.  Script  refers  to  the  JxK  complex  product 
space.  Superscript  ‘H’  denotes  the  complex  conjugate 
transpose.  Let  vec(Z)  be  the  operator  which  constracts 
a  column  vector  by  stacking  columns  of  the  matrix  Z. 
Finally,  ‘0’  denotes  the  Kronecker  product  {See  [4]). 

n.  A  Subspace  GLRT  Detection  Model 

Let  X  represent  an  N  by  L  matrix  of  data  formed  by 
concatenating  L  snapshots  of  the  A^'-dimensional  data 
observations.  The  snapshots  are  assumed  to  be  station¬ 
ary,  statistically  independent,  colored,  zero-mean,  mul¬ 
tivariate  Gaussian  distributed  noise  with  an  additive,  de¬ 
terministic  signal.  The  covariance,  R^,  of  each  snap¬ 
shot  is  assumed  unknown,  and  the  form  of  the  signal  to 
be  detected  is  given  by  E[X]=ab^C  with  ae  (C^, 
C  e  known,  and  b  €  is  an  unknown,  non¬ 
stochastic,  complex  matrix  of  regression  parameters. 
Hence  the  measured  data  array  is  modeled  as  vec(X)  ~ 
N(vec(ab^C),  R3j0 1^).  This  is  a  generalized  multi¬ 
variate  analysis  of  variance  model  (GMANOVA)  [2]. 

A  subspace  transformation  T  e  is  a  fixed  lin¬ 

ear  mapping  from  an  A-dimensional  space  into  a 
smaller,  P-dimension  space.  The  columns  of  T  are  as¬ 
sumed  to  be  linearly  independent.  Methods  for 


choosing  T  are  given  in  [1]  and  [2].  The  transformed 
data  Z  s  X  e  C  ^  ^  is  in  the  same  parametric  family : 

vec(Z)  ~  N(vec(THab^C),  (T»R^T)  01^).  (1) 

To  yield  a  meaningful  test,  we  require  that  T  be  chosen 
such  that  T^a  5*0.  The  GLRT  under  this  distribution 
can  be  expressed  as  [4;  p.23] 

aHT(ZPcxZ»)-'T"a  ^ 

aHT(zz'')''T''a  5^  ° 

where  Pcisl£-C^(CC^)~*C,  and  where  we  require 
L>M+N  in  the  full  data  space  (when  T  =Iyy)  and 
L>M+P  in  a  P-dimensional  subspace.  The  PD  and 
PFA  of  (2)  are  known  in  closed  form  and  are  given  in 
[1]  and  [2].  The  PFA  is  a  function  only  of  the  test 
threshold  ("g  and  dimensionality  parameters  L,  M,  and 
P.  The  PD  is  a  function  of  these  parameters  as  well  as 
the  non-adaptive  subspace  SNR 

SNRj  =  (b»CC”b)(aHT(T»R,T)"’T«a).  (3) 


m.  Signal  Mismatch 


Signal  mismatch  results  when  there  is  imperfect  knowl¬ 
edge  of  the  vector  a.  Hence  signal  mismatch  is  an  issue 
of  primary  concern  to  a  detector’s  robustness.  This 
section  will  show  that  subspace  processing  reduces  the 
effect  of  mismatch  between  the  assumed  signal  vector  a 
and  a  trae  signal  vector  a. 

The  hypothesis  test’s  performance  depends  upon  the 
signal  vector  only  through  the  non-adaptive  SNR  pa¬ 
rameter  (3)  used  in  the  PD,  and  then  only  through  the 
term  T%.  The  PD  is  a  monotonically  increasing  func¬ 
tion  of  SNR.  The  actual  performance  reduction  result¬ 
ing  from  using  the  assumed  subspace  signal  T^a  in¬ 
stead  of  the  true  quantity  T^a  depends  upon  the  angle 
between  these  two  vectors.  Kelly  [3]  shows  that  the 
true,  full  space  (T  =  I^),  non-adaptive  SNR  parameter 
SNRii^  =  (b^CC^b)(a^Rx*3)  is  attenuated  by  an  amount 


a“R*-'a| 


(a»R,-'a|aHR,-ia) 


(4) 


due  to  mismatch  between  the  assumed  signal  model  a 
and  true  signal  vector  3.  Here,  cos^(0yv)  represents  the 
cosine  squared  of  the  angle  between  a  and  a  in  an  ap¬ 
propriate  inner  product  space.  This  quantity  is  less  than 
one  whenever  a 5* a.  It  is  easily  shown  that  the 
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subspace  non-adaptive  SNR  (3)  is  attenuated  by  a  simi¬ 
lar  amount 

cos2(dp)=  ^ 

|a”T(T”R*T)~'T”a| 

(a“T(T«R,T)~^T“a)(aHT(T”R*T)“^T”a) 

The  relative  mismatch  performance  of  the  subspace  de¬ 
tector  is  as;sessed  by  comparing  cos2(djB)  to  cos^dff). 
For  instance,  if  cos2(6*^)  is  smaller  than  cos2(6|p),  then 
the  fiill-space  SNR  loss  is  greater  than  the  subspace 
SNR  loss. 

To  compare  (4)  with  (5),  begin  by  using  the  singular 
value  decomposition  (SVD)  of  R^2X  =  UEV^.  The 
first  P  columns  of  U,  denoted  by  {ui,  U2,  U3,  •••,  up), 
form  an  orthonormal  basis  for  the  range  space  of 
I^h/2t  jjjg  remaining  N-P  colunms,  denoted  by 
(up+i,  UP4.2,  •••,  u^},  form  an  orthonormal  basis  for  its 
orthogonal  complement.  Let  d  s  U^(Rx'/2a)  be  the  co¬ 
ordinate  representation  of  the  mismatched  quantity 
R;V2  a  in  this  new  basis.  The  coordinates  for  R^'/^a 
can  be  separated  into  two  sets  g  and  h  such  that 
jjHj  jjj  particular,  the  P  coordinates  of 

R"V2a  in  the  range  space  of  Rx/2T  are  given  by 
g  =  [ui,  U2,  ••*,  up]^(Rx*/2a),  and  the  (N-P)  coordi¬ 
nates  of  Rx^/2a  in  the  null  space  of  R^2T  are  given  by 
hs^Up+i,  Up+2» UAf]^(Rx*'^2a).  A  similar,  true 
quantity  Rx^/2a  has  a  coordinate  representation  defined 
bya  =  UH(R;V2a)  in  the  same  new  basis.  As  was  true 
above,  3  can  be  separated  into  two  sets  g  and  fi  such 
thata»  =  [gH  fi»]. 

Using  these  definitions,  it  is  easily  shown  that: 


Figure  2:  The  Subspace  Angle  Op  is  Smaller. 
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COs2(0p]  = 


(a”a)(d“d) 


g”g  ^ 


(g”g)(g”g) 


(6) 


With  these  definitions,  the  full  space  angle  cosine  can 
be  rewritten  as  follows: 


cos2(0^)  = 


g^g+h”h  ^ 


(g”g+h”fi)(gHg+h“h) 


g“g 

2 

-  h"h 

2 

+  2Re 

(gHg)(fi“h)) 

g  fi 

f] 

(|g”gh 

.|S"h| 

2 

(uNfiP! 

(sP-php) 

where  we  have  used 


(7) 


Re((|"g)(fi"h))£|Re((g"g)(fi"h))| 

^Re(|(|"g)(fi"h)|)  =  |(g"g)(B"h)|<|gHg|S%|. 

The  transformation  T  is  constrained  to  span  the  as¬ 
sumed  signal  a.  Therefore,  the  term  |  g  p  is  greater  than 
zero.  If  the  assumed  a  is  not  perpendicular  to  the  true 
3,  then  the  term  |  g  p  is  also  greater  than  zero.  The  goal 
in  designing  a  subspace  transformation  T  is  to  make  the 
non-adaptive  SNR,  (3),  approach  the  upper  bound 
SN/?{^  (see  [2]).  This  upper  bound  is  reached  when 
R;V2  a  lies  in  the  range  of  columns  of  Rx/2T.  For  such 
a  well  designed  T,  h  =  0.  In  this  case. 


cos 


-(M  ^ 


g”g 


gHg.2 


(8) 


g^' 


=  COs2(0p) 


with  equality  when  Rx'/2  a  also  lies  in  the  range  of 
Rx'^2X.  By  symmetry,  if  Rx*/2  g  lies  in  the  range  of 
Rx/2T,  then  fi  =  0,  and  again  cos2(0^)  <cos2(0p).  A 
perturbation  argument  shows  that  subspace  detection 
remains  more  robust  than  full  space  detection  under 
mismatch  when  either  Rx'^2  a  or  Rx''^2  g  lies  close  to 
the  range  space  of  Rx/2T.  Thus  if  T  and  a  are  chosen 
reasonably  well,  then  cos\6f^)  <  cos^(Op),  and  the  sub¬ 
space  SNR  loss  is  less  than  or  equal  to  the  full  space 
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SNR  loss  resulting  from  a  mismatched  signal.  Figure  2 
illustrates  how  the  angle  between  vectors  is  reduced  by 
the  subspace  mapping. 

Not  only  does  a  subspace  detector  loose  less  SNR  to 
signal  mismatch,  but  the  loss  of  SNR  can  affect  detector 
performance  to  a  smaller  degree.  This  effect  is  illus¬ 
trated  in  the  following  section. 


IV.  Examples 

To  demonstrate  the  effect  of  mismatch  loss  on  SNR, 
consider  a  ten-dimensional  system  (N=  10)  with  an  as¬ 
sumed  steering  vector  a=[l,0,0, ....  a].  For  simplic¬ 
ity,  assume  that  the  noise  is  white  (Rx=Iio)  that 
the  subspace  transformation  T  compresses  data  by 
throwing  away  data  from  the  last  six  sensors.  This 
simple  transformation  is  non-optimal  because  it  know¬ 
ingly  looses  SNR  by  throwing  away  signal  energy  (a) 
from  the  last  sensor.  However,  it  offers  a  simple 
demonstration  of  a  situation  where  a  does  not  lie 
in  the  range  of  columns  of  Rx-^T.  Let  the  true  steering 
vector  a  be  perturbed  from  a  by  a  Gaussian  distributed 
random  vector  with  covariance  Figure  3  plots 

the  mean  of  the  difference  in  mismatch  loss  (cos^(6p)- 
cosH^n))  over  30,000  trials  as  the  amount  of  signal 
amplitude,  a,  outside  of  the  range  of  R^2T  increases. 
Also  plotted  in  figure  3  are  the  percentage  of  trials 
where  subspace  SNR  loss  was  greater  than  the  full 
space  SNR  loss.  Note  that  even  when  one  half  of  the 
signal  model  lies  outside  of  the  range  of  rJ^2T,  i.e. 
a=l,  the  SNR  loss  due  to  mismatch  in  the  subspace 
detector  is  less  than  that  for  the  full  space  detector  in 
more  than  70%  of  the  random  trials.  If  the  amount  of 
signal  amplitude  outside  R^/^T  is  small,  then  subspace 


Figure  3:  Mean  of  mismatch  loss  differences  and 
cases  (%)  with  inferior  subspace  performance. 
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Figure  4:  Full  and  Subspace  Performance. 

SNR  loss  is  nearly  always  less  than  the  full  space  loss, 
as  predicted  by  the  analysis  in  the  previous  section. 

Now  consider  figure  4.  This  figure  depicts  the  PD 
as  a  function  of  non-adaptive  SNR  in  the  absence  of 
signal  mismatch.  Fifteen  snapshots  of  data  (L=  15)  are 
assumed  with  N=  10  and  a  PFA=  10“^.  The  subspace 
results  assume  that  a  transformation  from  ten  to  four  di¬ 
mensions  leads  to  an  SNR  loss  of  3  dB;  i.e.  the  sub¬ 
space  SNR  is  3  dB  less  than  the  values  depicted  on  the 
x-axis  as  “System  SNR”.  If  the  system  SNR  were  27 
dB  and  the  full  space  mismatch  resulted  in  a  4  dB  loss, 
then  the  full  space  PD  would  decrease  from  0.997  to 
0.508.  Because  the  subspace  mismatch  is  less  than  4 
dB,  the  corresponding  subspace  PD  decreases  from 
1.000  to  an  amount  still  greater  than  0.998. 
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